In a multiplex fluorescent slide of a tissue specimen, different nuclei and tissue structures are simultaneously stained with specific fluorescent dyes, organic fluorescent dyes, or fluorescent counter stains, each of which fluoresces in a different spectral band, while generally overlapping in the 400 nm-850 nm spectral range. More recently, quantum dots are widely used in immunofluorescence staining for the biomarkers of interest due to their intense and stable fluorescence. On a typical multiplex slide, a nuclei marker, for example, a 4′,6-diamidino-2-phenylindole (DAPI) fluorescent stain (i.e., a blue stain), is used along with the quantum dots. However, other nuclei counter stains may be used, such as, for example, other fluorescent counter stains. The slide is then imaged using a multi-spectral imaging system (such as a fluorescent microscope system that is coupled to a camera or a scanner; or a whole slide scanner). Each channel of the imaging system corresponds to a spectral narrow-band filter. The multi-spectral image stack produced by the imaging system is therefore a mixture of the underlying biomarker expressions, which, in some instances, may be co-localized.
In brightfield unmixing, the nuclei and tissue structures are stained with hematoxylin and eosin (H&E) or IHC staining. The slide is then scanned with the bright-field scanner equipped with COD color camera and finally the RGB image is acquired. Similarly to the multi-spectral image analysis, the reference colors of the RGB image are obtained from the scanned single stain bright-field images.
Taking the multi-spectral image unmixing as an example, to identify the individual constituent fluorescent dyes for the biomarkers and the proportions they appear in the mixture, spectral unmixing is used to decompose each pixel of the multi-spectral image into a collection of constituent spectrum endmembers (or reference spectra) and the fractions of their intensity contributions in the multi-spectral image from each of them. The reference spectrum for a stain corresponds to the emission spectral signature for the particular stain (e.g., fluorescent dye), when the stain is irradiated with spectra, for example, light of varying excitation wavelengths. The amount of endmember contribution is also referred to as abundance, and corresponds to a pixel value in the unmixed image, for example the grayscale pixel values in the DAPI or quantum dot unmixed images.
Accurate spectral unmixing of fluorescent images is clinically important because it is one of the key steps in multiplex histopathology image analysis. Several techniques have been proposed for spectral unmixing in the field of remote sensing, for example. One popular approach is solving for the abundances given the reference spectra within the non-negative least square (NNLS) framework. In this case, accurate estimation of the endmember contributions requires precise knowledge about the reference spectra. Unlike the applications in the domain of remote sensing where different objects such as grass and rocks can be easily identified from the scene, the biomarkers are often co-localized in histopathology fluorescent images, therefore making it difficult to extract the endmember spectra from the image and solve for the abundances. While the narrow-band reference spectra for quantum dot or nanocrystal biomarkers can be precisely measured from single stained control slides, broadband signals such as DAPI and tissue auto-fluorescence (AF) are slide-specific and exhibit variation between images and slide specimens. The broad-band spectra overlap with the narrow-band spectra, making the accurate estimation of the quantum dot abundances (i.e., the unmixed images) even more difficult. In addition, part of the DAPI spectrum may be wrongly estimated as AF because the spectral signatures of DAPI and AF are similar to each other, which may lead to an erroneous estimation of the nuclear component.
To simultaneously estimate the reference spectra, as well as the endmember contributions, non-negative matrix factorization (NMF) is used widely for blind spectral unmixing. However, due to the non-linearity of the problem, this method is not guaranteed to converge to a physically meaningful and relevant solution. This is especially relevant for the DAPI estimation because it is possible for NMF to worsen with every iteration for the same reason that the DAPI spectrum can be confused with other similar reference spectra, such as AF. NMF automatically solves for the reference spectral matrix, however the algorithm is not able to identify which reference spectrum corresponds to DAPI and which one corresponds to AF. Additional frameworks proposed based on the orthogonality assumption (i.e., the assumption that the reference spectrum of DAPI is orthogonal to that of quantum dots) of the endmember do not yield meaningful results for real data. Accurate DAPI unmixing is of great clinical importance because it is the most common nuclear stain. Moreover, nuclei detection serves as a first step in digital pathology image analysis, with further analysis tasks being based on the reliable identification of cell nuclei. Thus, there is a need for precise unmixing results. In the case of bright-field images, hematoxylin plays an equivalently important role as DAPI in multi-spectral images. Hence, the correct unmixing of hematoxylin is also very important.